Unit testing, discussed, for example, in the context of computer software development methodologies such as extreme programming (“XP”), has become a commonly-used technique. Developers write and maintain automated tests coded in the same programming language as production code. These tests typically are considered to be part of the code, and are included in revision control. The tests verify functionality, and also serve to document code by explicitly describing exemplary use of the functionality and expected results. Tests may be added or expanded upon to check for bugs and verify bug fixes. A typical unit test for a method may include code to set up the data used for testing (sometimes referred to as a fixture), a call of the method under test with the test data, a comparison of actual results with expected values, and tear down of the test data. Testing may be supported by testing frameworks, such as JUnit for JAVA, and NUnit for .Net, for example, which provide functionality to accomplish these tasks.
Some XP proponents suggest running tests on code every time it is changed. For example, a build tool may be configured to compile a file into a jar (in the case of Java code, for example) and then automatically run unit tests on the compiled code.
For example, the Ant build tool, available from the Apache Software Foundation, is platform independent, and is intended to be configured with a configuration file written in XML. The Ant configuration file describes a tree of targets, each target including one or more tasks that may be executed. For example, a task may be to create a directory (e.g., “mkdir”), perform a Java compile (e.g., “javac”) and so on. Ant includes a number of built-in tasks, and also is extensible. One task that may be performed with Ant is to execute tests within the JUnit testing framework. The build tool thus can be configured to automatically run unit tests on code that has changed.
In the context of complex web applications, it can be difficult to test code that is dependent on external server software applications. Examples of such server software applications are web servers (e.g., Apache Tomcat, Microsoft Internet Information Server) and database servers (e.g., MySQL database, Oracle 7, PostgreSQL). Test code that makes optimistic assumptions about the presence or state of any external resources can cause non-deterministic behavior in test outcomes. This is even more true if the test code relies on the presence or state of a particular server. While it may be possible in some cases to perform some simple tasks (e.g., create a directory, copy a database table) in the context of a testing framework or conventional build tools, it is frequently the case that it is not practical to administer and configure a server software application. As such, it may be difficult to automate such testing.
Jakarta Cactus, from the Apache Software Foundation, for example, provides extensions to the JUnit testing framework for servlets, which may be Java code that runs as a service. Cactus is intended to allow testing of the servlet by creating a mechanism to set up and tear down the servlet so that unit tests may be run on the servlet. Although beneficial for developing servlets, Cactus does not address starting or configuring a server software application that the servlet depends on.
One solution to this problem is the development of a “mock object” to test the behavior of other objects. A mock object may act as a faux implementation of an interface or class that mimics the external behavior of a true implementation. See http://www.mockobjects.com. Although mock objects may be helpful, it can be difficult to construct a mock object that sufficiently simulates the external behavior of a server software application to act as a useful substitute for testing.